Papers with Spearman correlation
BMX: Boosting Natural Language Generation Metrics with Explainability (2024.findings-eacl)
Copied to clipboard
| Challenge: | Modern language model (LM) based natural language generation evaluation metrics achieve astonishing results in grading machine generated sentences like humans would. |
| Approach: | They use word-level explanations to convert a word- level score into a segment-level score and combine this segment- level with the original metric to obtain a better metric. |
| Outcome: | The proposed metrics improve across MT and summarization datasets while achieving small improvements on machine translation. |
Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used as evaluators but the reliability of the outcomes remains a challenge. |
| Approach: | They show that LLM judge outputs are highly sensitive to pre-defined score ranges and that similar biases exist among models from the same family. |
| Outcome: | The contrastive decoding of LLM judge outputs achieves 11.7% relative improvement in Spearman correlation with human judgments, averaged across score ranges. |
Co-Eval: Augmenting LLM-based Evaluation with Machine Metrics (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing LLMs suffer from biases and misalignment due to limited functional understanding and knowledge gaps. |
| Approach: | They introduce a framework that leverages a criteria planner model and optimized machine metrics to enhance the scalability and fairness of LLM-based evaluation. |
| Outcome: | The proposed framework reduces biases and improves alignment with human preferences, with gains of up to 0.324 in Spearman correlation. |
Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Semantic similarity between two sentences depends on the aspects considered between those sentences. |
| Approach: | They propose a Conditional Semantic Textual Similarity task which measures the similarity between two sentences under a specified condition. |
| Outcome: | The proposed method improves Spearman correlation by 5.4% by training a supervised model on the re-annotated dataset. |
CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are outdated or incorrect over time due to unintended ripple effects that propagate even to the hidden space. |
| Approach: | They propose a lightweight representation-level technique to identify where ripple effects may occur by detecting entanglement between facts using forward activations from a single intermediate layer. |
| Outcome: | The proposed method achieves 62.2% improvement in Spearman correlation with ripple effects while being 2.74 faster and using 2.85 less peak GPU memory. |